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Abstract 



Conference evaluation, as with other forms of test 
and measurement, requires adequate resources, both per- 
sonal and financial. In many Instances various individ- 
uals involved in a conference perceive its values 
differently. This is therefore one of the ma i n obj ec- 
tives in conference evaluation. 

The evaluation process is different things to 
different people. Program planners, participants and 
sponsors need to be considered when designing evaluation. 
Conference evaluation is undertaken to improve the oper- 
ation of an organization, or perhaps to determine the 
relevance of a program to aspirations of participants. 

Designing an evaluation warrants consideration to 
the fol lowing: 

-What to evaluate 

-Relating objectives and evaluation 
-Management of the evaluation process 
-Who should conduct evaluation 
-What does evaluation accomplish 
-What are symptoms of inadequate 

evaluation and why is it neglected 
-V/hat are obstacles and hurdles to 

evaluation 
-V/hc,t are common methods and techniques 

employed in conference evaluation. 

Evaluation must be considered in initial planning 
and requires total commitment from planning committees, 
and/or sponsor , participants and employers. A lack of 
enthusiasm and belief in its value will force it to 
appear superficial . 
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JUTRODUCTION 

Teachers jnd Instructors devote substantial periods of time developing 
tests, examinations, and diverse methods for evaluating their efforts. Con- 
ference planners, in like manner, concentrate effort to "evaluation" to im- 
prove and analyze the programs for their older and more mature conferees. 

Conferences, if operated efficiently, can bring the greatest satisfaction 
to a coordinator. Likewise, a poorly evciluoted conference does little to 
support the planning process. Consequently, it is vital that a conference 
director or coordinator recognize that evaluation is continually in operation 
,ind guard against possible weaknesses. 

Of special note to this paper are the history and growth of conferences, 
defining evaluation, its purposes, who it assists, and how results are utilized, 

HISTORY AND DEFINITION 

An origin of the conference method is unknown but the oldest method of 
teaching, as presently perceived, was accomplished by a conference. The 
Socratic method was in reality a form of conference discussion. Historic 
vocational education movements also were, in essence, conference methods, the 
most significant being the work o"f Froebel and Pestalozzi. In practice, the 
conference is bos leal ly a vocational education venture, seeking Improvement of 
participants' competencies or development of untapped personal resources. 

According to H. Leroy Harlow (I963), 

A conference in its purest sense is an informal, yet planned 
meeting of experienced people with related interests and 
common problems, who through discussion pool their Ideas 
and strive for a solution. 

Obviously, conferences are conducted for diverse purposes; the most predominant 
being problem solving, information sharing, policy development, resolution of 
conflict, discussion of a mutual area or specific problem. 
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Familiarity with conference planning is essential ond includes: 

The conscious and deliberate guidance of thinking to create 
logical means for achieving commonly agreed upon goals. 
Planning always and inevitably sets priorities and calls^ 
for the vaJuc judgments. The alternative to plan is not 
plan (Treacher, 1950). > 

As compared to the systematic and organized previous definition, confer- 
ence plonning often is undertaken by a more informal approach defined as: 

Unorganised, unplanned v/ork of- a.. few individuals in 
assuming to recognize the wants and desires of po- 
tential participants. Perhaps a few participants will 
attend, but quite often by happenstance. Perhaps a 
few objectives will be met, but often by chance. 

This second approach is based on assumptions of "what is best" for participants. 

Supplementally, speakers and topics are selected with no consideration of 

desires or needs of participants. Disadvantages are numerous and evident; 

comprehensive evaluation is impossible; a majority of potential participants 

are ignored ahd quite often the conference is a financial blunder. 



CONFERENCE COMPONENTS 

Aspirations of the conference planner, along with objectives of a 
conference, are the major components of conference planning. The inclusion 
of these components is vital to potential participants and the planner, A 
conference should aspire to satisfy expectations and objectives of participants. 
An additional major component common to planning and participants is evaluation. 

Incorporating aspiration of participants in any conference must include: 

1. Identifying a common interest or need. This is vital in planning 
procedures, for it is necessary to generate a relevant theme. 

2. Developing topics, issues, or questions which the conference will 
discuss, 

3. Establishing and clarifying objectives, 

h. Selecting appropriate speakers, films, materials, etc., which 
assist participants in meeting stated objectives. 
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5. Evaluating program, through various methods and techniques, to 
determine conference success* 



THE EVALUATION PROCESS 

Conclusions, pro or con, will be fabricated regarding personal satis- 
faction from attending a conference. Participants, speakers, and coordinators 
^will decide, according to individual criteria, conference effectiveness. It 
is inappropriQ.te to ask, "Shall we evaluate?" A more appropriate question 
IS, "How and to what extent should we evaluate to improve the conference?" 

Informally, conference evaluation is what participants say to each other 
about the speaker as they leave a workshop session and return to their occu- 
pations. What a participant reports to his boss upon returning to the job is 
evaluation. Evaluation is what people state on the evaluation instrument; it 
is their professional behavior three months or a year following the conference. 
Evaluation in total is an appraisal by participants, speakers an6 planners at 
the conclusion of a conference. 

A formal definition, as stated by Knowles (1970), applicable to many 
phases of adult education states: 

...evaluation has two principle purposes: l) Improvement 
of organization operation, including such aspects as its 
planning process, structure, decision-making procedure, 
personnel, physical facilities, public relations, and 
administration and management* 2) Improvement of the 
program, including such aspects as objectives, clientele, 
methods and techniques, materials and quality of learning 
outcomes . 

Evaluation can also be used for such secondary purposes 
as defense against attack, justification for expansion, 
support for the status quo, boosting of morale, personnel 
appraisal and promotion, and institutional reorganization. 

Nathan C. Shaw (I969) emphasized collection and analysis of data in terms 
of conference effectiveness. 
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Logical queries emerge: What is evaluated in a conference? For what 
reasons do participants attend a conference? Participants aLLcnd for a number 
of reasons, the lea^st of which may be professional improvement. It may be 
to improve their knowledge of a subject, to receive information pertaining 
to their job, or because a superior has encouraged attendance and is paying 
all expenses. It may be an attempt on the part of the participant to '*get away 
from the present j^o responsibilities or from the spouse. It could be to 
"search'^ for a new employer while being financed by the present boss. Reasons 
for attendance are any number of social reasons, excluding job improvement. 

This diversity for attendance creates problems for conference evaluation. 
The evaluator must now be concerned with such aspects as are included in the 
"happi less" area rather than programmatic concerns. Were you "happy'* with 
motel-hotel, transportation, food, physical facilities; were chairs comfortable 
lighting adequate, etc.? These items are of minor concern, for they seem in- 
significant in terms of the purpose of a conference. Dissatisfaction with 
these "happiness" items .could reduce attendance at a future conference. Their 
evaluation is useful, keeping in proper perspective the vyeight of their 
measures . 

WHAT TO EVALUATE 

What is evaluated in a conference? Obtaining information consistent with 

stated objtw'ives is an essential first step. Evaluation from this perspective 

requires varied operations* Lindeman (1953) analyzes the consequence as: 

*1, Stating objectives of a group or planning committee in specific 
terms so that evidence of the degree to which objectives are 
being achieved can be obtained. 

2, Securing evidence of the degree to which objectives are being 
achieved. 

3, Securing facts regarding what is being done to achieve objectives. 
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h. Developing ideas about what factors might be helping or hindering 
achievement of objectives. 

5. Developing and trying out methods of remedying weaknesses in 
conferences. 

In relation to the first consequence, and outlined by Hager (1962), an 
objective attempts to accomplish: 

...an intent communicated by a statement describing a 
proposed change in a learner--a statement of what the 
leorner is to be like when he successfully completes 
a learning experience; Objectives of a training program 
must denote measurable attributes observable in the grad- 
uate of the program, or otherwise it is impossible to 
determine whether or not the program is meeting the 
objectives. 

Absence of clearly stated objectives complicates evaluation. Consequently, 
it is difficult to select speakers, materials, and a method of instruction. 
Precisely stated objectives assist the participant, for they allow self- 
assessment of individual progress related to anticipated needs. Advance dis- 
semination of conference objectives allows participants to decide the benefit 
of attendance. If possible, this information should be included with pre- 
registration material. 

Mager (1962) states that essential characteristics of objectives are: 

F i rs t , they identify the terminal behavior by name; we 
can specify the kind of behavior which will be accepted 
as evidence that the learner has achieved the objective. 
Second , try further to define the desired behavior by 
describing the important conditions under which the be- 
havior will be expected to occur. Thi rd , specify the 
criteria of acceptable performance by describing how well 
the learner must perform to be considered acceptable. 

It is important to emphasize that objectives define performance characteristics 

and anticipated accomplishment. 

In addition to stated objectives, latent objectives offer potential and/or 

resistance to evaluation. Participants have objectives for attending. There 

are social objectives that deal with the relationship of the conference to 
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personal improvements. Non-eJucotional objectives seek to improve the organi- 
zaUonal image or membership. Speakers have objectives; ihey may include dis- 
cussion of a new theory or participants' reaction to a point of view. Ex- 
cessive effort by a planning committee on latent objectives should be discour- 
aged. A poorly planned conference could satisfy latent objectives and show 
little regard for educational objectives. Conference planners, when developing 
educational objectives, must react to the following: 

1. Ar^ objectives specific enough to serve as a base for evaluation? 

2. Are objectives perceived similarily by all concerned? 

3. Do objectives correspond to needs of participants?" 

^. How will participant behavior be modified if objectives are met? 

5. Are objectives limited and can they be accomplished in time allotted? 

To insure relationship of objectives to evaluation, a full-time evaluator 
IS useful and often essential. This individual should have responsibilities 
in initial planning of the conference, as well as relating objectives to the 
nature and scope of evaluation design. 

RELATING OBJECTIVES AMD EVALUATION 

A difficulty encountered when reacting to the five concerns above is 
failing to state objectives specifically. Ambiguous objectives complicate 
measuring change in participants* behavior. The initial link between objectives 
and evaluation occurs when the planning committee outlines what is to be 
accomplished in the conference. Anticipated changes may involve personal 
characteristics, knowledge, or perceptions and attitudes. Changes may occur 
with participants' role performance; being a better worker, being more coop- 
erative and understanding of the job. Of necessity, it is important for eval- 
uator, planning committee and participants that logic for establishing 
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conference objectives be clearly understood and all are knowledgeable of 
possible-outcomes . 



MANAGEMENT OF THE EVALUATION PROCESS 
The following questions remain: 

1. Who should be assigned to conduct evaluation? 

2. V/hat does evaluation accomplish, what are its outcomes, and how 
are results utilized?. 

3. What are symptoms of Inadequate evaluation and why is it often 
neglected? ^ 

h. What are common obstacles or hurdles to evaluation? 

5. What are methods and techniques utilized in conference evaluation? 



WHO SHOULD EVALUATE 

Ideally, one individual or group should be assigned to evaluation. Costs 
related to this model are expensive and therefore seldom implemented. There- 
fore, every individual related to conference planning or implementation in a 
position to make judgments about a conference should be involved. Concerning 
participants as evaluators, Lindeman (1953) points out: 

If an individual takes part in discovering the effects 
of an activity in which he has a part and then helps 
to work out plans for improving the activity, he i' 
J i ke 1 y to : 

...have confidence in the evaluation process, 
...understand what effects his own work has on 

the success of the activity, 
...originate or accept suggestions, for improving 

hi s work, and 
...be willing to make changes or to learn to make 

changes that seem desirable. 

In addition to participants, other evaluators might include: speakers at 
the conference, employers of participants, community representatives, and 
special consul tants. 
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Each group perceives a conference os accomplishing purposes unique to 
itself and thus each group perceives evaluation differently. Including all 
groups will provide diverse reactions and will Incorporate aspects from 
conference pre-planning to application of presented information. 

WHAT DOES IT ACCOMPLISH 

Evaluation is expensive and time consuming but yet it is vital and 
should not be overshadowed for cost factors. It will quite easily reimburse 
initial planning cost. A basic accomplishment is assessment of anticipated 
results and outcomes. Evaluation should assist in planning a future conference. 
Non- programmatic items such as accommodations, meals, transportation, 
physical facilities can be enhanced through evaluation. Evaluation will" 
Ident'fy those areas in which a conference \^as strong, as well as areas in 
need of improvement. A majority of areas in need of improvement indicates 
additional planning and development are in order. 

Possible results from evaluation include an altered format from lecture 
to discussion, utilization of audiovisual and handouts; or objectives could 
be altered to respond to changing needs and interests of participants. In- 
formation from evaluation could promote the sponsoring organization. Perhaps 
the organization is unav^are of problems and a conference evaluation could 
uncover these. Thus, the organization may determine initial conference 
planning to be inadequate and insufficient and recognize a need for additional 
planning. 

A final act of accomplishment is reporting conference proceedings, by 
evaluators, to the general public and to participants. This reporting may 
include recommendations for futura action or a summation of findings from 
the evaluation. An important concern is that it is unbiased and fair. 
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SYMPTOMS OF INADIiQUATE EVALUATION AND WHY IT IS NEGLECTED 

Symptoms of inadequate evaluation are diverse. According to Shaw (I963), 

major symptoms of inadequate evaluation include: 

Low re-enrol Imcnts in re-occurring conferences, 

Difficulty in attracting participants from specific 
populations. 

A high dropout rate durjng a conference. 

Complaints from employers of participants regarding 
obsolescences of knowledge or skills presented at 
a conference. 

The inability of participants to apply what has been 
presented to an actual situation. 

A lack of financial support from policy makers regarding 
budget requests and proposals for future conferences. 

Symptoms of inadequate evaluation often rel;ate to its neglect. Reasons 

for this neglect might include: 

1. A feeling that there is nothing worth evaluating. 

2. Program objectives were poorly constructed and difficult to 
evaluate. 

3. The conference sponsor is lethargic and does not wish to commit 
the required resources, 

^. The conference couid be criticized if results are insignificant 
and little -/alue is realized. 

OBSTACLES AND HURDLES TO EVALUATION 

The true value of a conference is its effect on participants. Conferences 
are to produce change in knowledge, attitudes, conduct, or thinking. A 
conference is not to control thinking but rather to influence thinking or 
actions of participants. A participant realizes reasons for attendance ^and 
will judge the success of a conference accordingly. Participants observe 
their colleagues and their changed behavior, make comparisons, and discuss 
the conference. Thus, behavior, comparisons, and discussions cause difficulty 
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in evaluation. It Is possible to cvahate satisfaction with physical arrange- 
ments and facilities and inquire if the conference has satisfied needs. It 
IS a laborious task to evaluate total behavior change, participant comparisons, 
and subse^l^^nt discussions. 

Evaluation is not synonymous with measurement for measurement involves 
comparison. Evaluation is often a one-time event and a base of comparison 
does not exi st. 

A third obstacle to evaluation is financial. To be comprehensive, 
evaluation requires a commitment from decision makers. Therefore, a frequent 
excuse is it costs too much. In reality, the true reason may be fear of 
uncovering shortcomings in conference planning. Hurdles for Evaluation (1953) 
discusses skepticism as an obstacle to evaluation. Misapplications of 
evaluation which create this skepticism are: 

1. Time devoted to evaluation is spent not in improving the program 
planning and performance, but in defending the program to 
appease outside interests which may be viewed as Jiostile 

to purposes and standards.. 

2. Evaluation may be utilized as a way to eliminate ^Mneff icient" 
workers, program areas, or to justify retrenchment in the 
budget. This area is especially applicable to a planning 
commi ttee. 

3. Perhaps participants have put serious effort and time into 
self-examination of what the conference has accomplished for 
them only to find the time wasted when the evaluation report 
Is filed c-jway and forgotten. 

A poor commitment of time to comprehensive evaluation only supports 
this skepticism and reinforces personal attitudes and threats. 

A final obstacle is unclear and non-specific objectives. A participant 
cannot evaluate a conference if objectives are not clearly stated and under- 
stood. In order to evaluate the value of a conference, he must perceive the 
change in behavior; this is accomplished through objectives. 
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Evaluation must be discussed and planned, must have support from policy 
makers, participants, and speakers, and must be utilized in an efficient and 
practical manner rather than filed and forgotten. 

METHODS A MD T ECHNIQUES OF CONFERENCE EVALUATION 

Methods and techniques for evaluating a conference involve initial 
construction of a plan. It is essential for evaluation to be included as 
planning proceeds. Planning includes clearly stated objectives. Thus, 
objectives might determine methods and techniques. Clearly stated objectives 
are easier to evaluate and construction of instruments to accomplish this is 
facilitated, 

Basic problems are encountered in development of instruments. Miller 
and McGuire (I96I) state the most common problems as being: 

1. Finding or constructing the most convenient and economical 
instrument that will elicit responses which bear close 
relation to the behavior that could be expected from a 
participant in a non-test situation. 

2. Considerable variation in the amount of change which the 
conference can reasonably expect to achieve due to varied 
experiences of participants. 

3. An enormous range of specific content involved in a con- 
ference and each participant may respond differently. 

^. Participants at a conference do not expect to be evaluated 
and thus may exhibit resistance to the idea. 

An evaluator, working with a planning cdmmittee, K.-^n best eliminate or 
minimize these problems. Being alert to problems in construction and implemen- 
tation of evaluation will be reduced and its effectiveness increased with em- 
ployment of evaluation staff. 

Validity and reliability are of concern to conference evaluation. The 
instrument or process must be valid and accomplish its intended purpose. It 
must be reliable and supply consistent data from participants. 
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evaluation occurs three distinct periods or in any combination of the 
three: l) before the conference, 2) during the conference, and 3) after the 
conference* Each period allows for specific aspects of evaluation. Utili- 
Mtion of pre-conference evaluation allows for pre-test and post-test sessions* 
This is often the most advantageous, for it allows a study of behavior prior 
to and subsequent to participation. 

Evaluation during a conference allows for periodic on-site checks of 
-attitude. V/ithin this period, material i'^ easily recalled by participants 
and they can state immediate reactions. 

Post-conference evaluation is most widely accepted. It allows partici- 
pants to evaluate the conference in relation to job situations and allows the 
opportunity to evaluate change in behavior as related to conference objectives. 
An employer's observation of behavior of the participrjnt following attendance 
IS also post-conference. 

Specific methods best lend themselves to specific periods. Methods most 
commonly employed are: 

Questionnaires sent to participants (Pre-test and Post-test) 
Sample interviews 
Follow-up local meetings 

Reports from part ic ipants ' local organ i ?:a t ions 

Reports from field staff 

Interviews with participants' colleagues 

Interviev's with participants' employers 

Reports of the evaluation committee 

Post -conference meeting with conference committees. 

Each method has advantages or disadvantages and variables such as number 

of participants, conference format, nature of material presented, and avaUable 

resources must be considered in selection of appropriate methods. 

Research, investigation, and experience will support the contention that 

no standard exists for selecting a. most acceptable method of evaluation. 

Material by Kempfer (ig'lS), Knowles (I966), and Uramech (I966) provides 
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examples of evaluation instruments appropriate for conferences, but only the 
evaluator can determine which system or method is. most appropriate. 

SUMMATION 

The conference method of Instruction and presentation in adult education 
is not questioned. A conference can easily be prepared and implemented, but 
the value of all preparation and planning is measured through effective cvalT 
uation. 

Evaluation must be considered in initial planning, and requires total 
commitment from planning committees and/or sponsors, participants, and em- 
ployees. A lack of commitment and belief in its value will force it to appear 
as superf i ci a 1 . 
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